A Broad Study of Pre-training for Domain Generalization and Adaptation

Deep models must learn robust and transferable representations in order to perform well on new domains. While domain transfer methods (e.g., adaptation, generalization) have been proposed across domains, they are typically applied ResNet backbones pre-trained ImageNet. Thus, existing works pay little attention the effects of pre-training tasks. In this paper, we provide a broad study in-depth analysis for adaptation generalization, namely: network architectures, size, loss, datasets. We observe that simply using state-of-the-art backbone outperforms baselines set Office-Home DomainNet improving by 10.7% 5.5%. hope work can more insights future research.